Goto

Collaborating Authors

 line graph





Chimera: State Space Models Beyond Sequences

Lahoti, Aakash, Marwah, Tanya, Puduppully, Ratish, Gu, Albert

arXiv.org Artificial Intelligence

Transformer-based deep learning methods have become the standard approach for modeling diverse data such as sequences, images, and graphs. These methods rely on self-attention, which treats data as an unordered set of elements. This ignores the neighborhood structure or graph topology of the data and requires inductive biases--such as position embeddings in sequences and images, or random walks in graphs--to incorporate topology. However, designing such task-specific biases requires significant effort and can introduce side effects that hinder generalization. We introduce Chimera, a unified model that directly incorporates data topology in a principled way, removing the need for domain-specific biases. The key idea is that state space models--which naturally do not require position embeddings--can be generalized to capture any graph topology. Our experiments show that Chimera achieves strong performance across language, vision, and graph domains, outperforming BERT on GLUE by 0.7 points, ViT on ImageNet-1k by 2.6%, and all baselines on the Long Range Graph Benchmark. We further propose algorithmic optimizations to improve Chimera's efficiency: (1) for Directed Acyclic Graphs, Chimera can be implemented as a linear-time recurrence; (2) for general graphs, a simple mathematical relaxation achieves Transformer's quadratic complexity without domain-specific heuristics. These results validate Chimera's core contribution and support the idea that data topology is a powerful inductive bias across modalities.






A pseudo-inverse of a line graph

Kandanaarachchi, Sevvandi, Kilby, Philip, Ong, Cheng Soon

arXiv.org Machine Learning

Graph perturbations are used to test robustness of algorithms. The expectation is that for small graph perturbations algorithm output should not change drastically. While graph perturbations are extensively studied in many contexts, they are underexplored for line graphs, where a line graph is an alternative representation of a graph obtained by mapping edges to vertices. But line graphs are increasingly used in many graph learning tasks including link prediction Cai et al. (2021), expressive GNNs Y ang & Huang (2024) and community detection Chen et al. (2019), and in other scientific disciplines Ruff et al. (2024), Min et al. (2023), Halldórsson et al. (2013). The reason that line graph perturbations are not commonly used is because the perturbed graph may not be a line graph. We introduce a pseudo-inverse of a line graph, which generalises the notion of the inverse line graph extending it to non-line graphs. The proposed pseudo-inverse is computed by minimally modifying the perturbed line graph so that it results in a line graph.


MultiRAG: A Knowledge-guided Framework for Mitigating Hallucination in Multi-source Retrieval Augmented Generation

Wu, Wenlong, Wang, Haofen, Li, Bohan, Huang, Peixuan, Zhao, Xinzhe, Liang, Lei

arXiv.org Artificial Intelligence

--Retrieval Augmented Generation (RAG) has emerged as a promising solution to address hallucination issues in Large Language Models (LLMs). However, the integration of multiple retrieval sources, while potentially more informative, introduces new challenges that can paradoxically exacerbate hallucination problems. These challenges manifest primarily in two aspects: the sparse distribution of multi-source data that hinders the capture of logical relationships and the inherent inconsistencies among different sources that lead to information conflicts. T o address these challenges, we propose MultiRAG, a novel framework designed to mitigate hallucination in multi-source retrieval-augmented generation through knowledge-guided approaches. Our framework introduces two key innovations: (1) a knowledge construction module that employs multi-source line graphs to efficiently aggregate logical relationships across different knowledge sources, effectively addressing the sparse data distribution issue; and (2) a sophisticated retrieval module that implements a multi-level confidence calculation mechanism, performing both graph-level and node-level assessments to identify and eliminate unreliable information nodes, thereby reducing hallucinations caused by inter-source inconsistencies. Extensive experiments on four multi-domain query datasets and two multi-hop QA datasets demonstrate that MultiRAG significantly enhances the reliability and efficiency of knowledge retrieval in complex multi-source scenarios. Large Language Models (LLMs) have achieved remarkable success in handling a variety of natural language processing tasks, attributable to their robust capabilities in understanding and generating language and symbols [1]. In knowledge-intensive retrieval tasks, Retrieval Augmented Generation (RAG) has become a standardized solution paradigm [2]- [4]. W enlong Wu and Haofen W ang contributed equally to this work. Bohan Li is the corresponding author .